CytoAtlas

Why independence correction matters: Pooling across tissues or cancer types inflates correlations through confounding. For example, GTEx pooled CytoSig median ρ (0.211) is 40% higher than the independence-corrected by-tissue value (0.151); SecAct shows +26% inflation (0.394 vs 0.314). All results in this report use the corrected values. For a detailed comparison of pooled vs independent levels, including inflation magnitude and finer cell-type stratification, see the Section 4.1 statistical supplement.

The key insight: CytoAtlas is not trying to replace DL-based tools. It provides a linear, interpretable readout: when CytoAtlas reports “IFNG activity is elevated in CD8+ T cells from RA patients,” the contributing signature genes and their weights can be examined directly in those cells.

PRIMARY independent level: The summary table above reports results at each dataset’s PRIMARY independent level — the aggregation level where samples are fully independent (each donor counted once). This ensures correlation statistics are not inflated by donor duplication. See the “Primary Level” column for each dataset’s level.

How “N Targets” is determined: A target is included in the validation for a given atlas only if (1) the target’s signature genes overlap sufficiently with the atlas gene expression matrix, and (2) the target gene itself is expressed in enough samples to compute a meaningful Spearman correlation. Targets whose gene is absent or not detected in a dataset are excluded. CytoSig defines 43 cytokines and SecAct defines 1,170 secreted proteins. Inflammation Atlas Main retains only 33 of 43 CytoSig targets and 805 of 1,170 SecAct targets because 10 cytokine genes (BDNF, BMP4, CXCL12, GCSF, IFN1, IL13, IL17A, IL36, IL4, WNT3A) are not sufficiently expressed in these blood/PBMC samples.

Stratified levels (GTEx by_tissue, TCGA primary_by_cancer): Correlations are computed within each tissue/cancer type (ensuring independence), then summarized across groups. N Targets counts unique targets at the “all” aggregate level. Finer per-tissue or per-cancer breakdowns are available in Section 4.3 below.

Why does SecAct appear to underperform CytoSig in Inflammation Atlas Main?

The total comparison includes ~1,170 SecAct targets vs 43 CytoSig targets, so differences in the non-overlapping targets drive the result. Two complementary tests separate this from matched-target performance:

Total comparison (Mann–Whitney U test): Compares the full ρ distributions of CytoSig (43 cytokine signatures) vs SecAct (~1,170 secreted protein signatures) using independence-corrected values. For GTEx/TCGA, each target’s representative ρ is the median across per-tissue/cancer values (median-of-medians); for other datasets, donor_only/tumor_only ρ is used directly. SecAct achieves a significantly higher median ρ in 5 of 6 datasets (GTEx: p = 4.76 × 10⁻⁴; TCGA: p = 2.85 × 10⁻³; CIMA: p = 3.18 × 10⁻²; scAtlas Normal: p = 1.04 × 10⁻⁴; scAtlas Cancer: p = 1.06 × 10⁻⁵). Inflammation Atlas Main is the sole exception (U = 14,101, p = 0.548, not significant) and the only dataset where CytoSig’s median ρ (0.323) exceeds SecAct’s (0.173).

Matched comparison (Wilcoxon signed-rank test): Restricts to the 32 targets shared between both methods (22 direct + 10 alias-resolved), each target serving as its own control. SecAct’s median ρ is consistently higher across all 6 datasets, reaching significance in 5 (GTEx: p = 3.54 × 10⁻⁵; TCGA: p = 3.24 × 10⁻⁶; CIMA: p = 2.28 × 10⁻²; scAtlas Normal: p = 3.54 × 10⁻⁵; scAtlas Cancer: p = 3.54 × 10⁻⁵). Inflammation Atlas Main is not significant (p = 0.141).

Inflammation Atlas Main is largely blood-derived; 99 SecAct targets show negative ρ only in this dataset while remaining positive in all other datasets. The “Matched” tab above restricts the comparison to the 32 shared targets, removing the effect of non-overlapping targets.

Stratified validation: Instead of aggregating tissues/cancers into a single median-of-medians, this view shows the CytoSig vs SecAct comparison within each individual tissue (GTEx) or cancer type (TCGA). Mann-Whitney U test (Total tab: all targets) and Wilcoxon signed-rank test (Matched tab: 32 shared targets) with BH-FDR correction across all strata within each dataset.

Key insight: On matched targets (23 pairs per GTEx tissue, 19–21 per TCGA cancer after expression filtering), SecAct wins direction in 29/29 GTEx tissues and 32/33 TCGA cancer types, with 23/29 and 31/33 reaching significance (q<0.05). The sole exception is Acute Myeloid Leukemia (Δ=−0.08, q=0.83). This consistency across 61 of 62 strata indicates the SecAct advantage holds at the individual tissue/cancer level, not only in aggregate. On total targets, SecAct wins in 28/29 GTEx tissues (21 significant) and 30/33 TCGA cancers (15 significant); the few CytoSig-favored strata (Brain in GTEx; Kidney Chromophobe, Ovarian, Uveal Melanoma in TCGA) are all non-significant. Because this comparison uses the same 23–32 matched cytokines, the difference reflects per-target signature performance rather than target count. The largest Δ values occur in Small Intestine (+0.55), Vagina (+0.34), Pancreatic (+0.34), and Head & Neck (+0.27); the smallest in Heart (+0.06), Skin (+0.06), Uveal Melanoma (+0.07), and Kidney Chromophobe (+0.10).

Cross-platform concordance: This section tests whether expression–activity relationships replicate across measurement technologies. For each tissue (GTEx) or cancer type (TCGA), we compute per-target Spearman ρ from bulk RNA-seq data and compare it to the same target’s ρ from single-cell pseudobulk data (scAtlas). Wilcoxon signed-rank tests (paired by target) with BH-FDR correction assess whether ρ values differ between platforms.

Key finding: Using all targets, SecAct shows significant bulk–pseudobulk differences in most strata (11/13 GTEx tissues, 5/11 TCGA cancers), while CytoSig shows almost none (1/13, 0/11). When restricted to the same 32 shared targets, both CytoSig and SecAct show no significant platform differences (0/13 and 0/13 for GTEx; 0/11 and 1/11 for TCGA). Matched and unmatched SecAct targets show the same per-target platform shift (mean |Δ| = 0.298 vs 0.302, Mann–Whitney p = 0.82), but SecAct’s ~1,000 paired targets per tissue provide 25× more observations than CytoSig’s ~40, allowing detection of a small systematic shift (Δ ≈ 0.03) that is not detectable with CytoSig’s sample size.

Key insight: 16 of 32 matched targets show mean ρ > 0.25 for both CytoSig and SecAct. For 11 additional targets, CytoSig averages near zero while SecAct achieves mean ρ > 0.2 (e.g., TWEAK: CytoSig −0.02 vs SecAct +0.44; LTA: −0.02 vs +0.53; HGF: +0.06 vs +0.58). Note: this section uses donor-level aggregation (matching Figure 5), not the independence-corrected median-of-medians used in §4.1–4.3. Select “SecAct” in the dropdown above to explore interactively.

Key insight: Mean |ρ| across 14 targets does not separate by platform: GTEx 0.26, TCGA 0.33, CIMA 0.26, Inflammation Main 0.44, scAtlas Normal 0.36, scAtlas Cancer 0.37. The universal tier (IL1B, TNFA, IFNG, IL6) shows ρ > 0.1 in all 6 datasets; context-dependent targets (IL4, IL17A) show sign changes and are absent from Inflammation Main.

Aggregation levels explained: Pseudobulk profiles are aggregated at increasingly fine cell-type resolution. At coarser levels, each pseudobulk profile averages more cells, yielding smoother expression estimates but masking cell-type-specific signals. At finer levels, each profile is more cell-type-specific but based on fewer cells.

Key insight: Beyond ~L2 annotation depth, % positive correlations approach 50% and signal/null ratio drops below 2×. Inflammation Main retains signal at L2 (553 samples/group) while scAtlas Normal shows weaker correlations at its shallowest stratification (22 samples/group). SecAct’s larger target pool (~1,170) shows higher retention than CytoSig (43 targets) in aggregate, but on matched targets the difference disappears.

Key finding: In PBMC-dominated datasets (CIMA, Inflammation Main), most direct correlations collapse after cell-fraction adjustment: CIMA retains only 29% of positive correlations (CytoSig) and Inflammation Main retains 43%, with median Δρ of −0.16 and −0.30 respectively. This suggests that donor-level correlations in these datasets are largely driven by cell-type composition rather than within-cell-type biology.

In contrast, scAtlas Cancer retains 97% of positive CytoSig correlations (median Δρ=−0.09) and 88% for SecAct (Δρ=−0.17). This is expected: tumor cells dominate the cell composition, so inter-donor variation in tumor cell fraction is smaller relative to within-tumor biological variation.

Key patterns in scatter data: IL1B is the most consistently well-correlated target across all 6 datasets and both signatures (ρ=0.37–0.77). CD40L and TRAIL show a CytoSig-specific sign reversal in CIMA (ρ=−0.48/−0.46) and Inflammation Main (ρ=−0.55/−0.54), while SecAct maintains positive correlations for the same targets (CD40LG ρ=0.14–0.71; TNFSF10 ρ=0.31–0.77 except scAtlas Normal −0.004).

HGF shows a large method difference: CytoSig yields negative or weak correlations in CIMA (−0.25), Inflammation Main (−0.30), and TCGA (0.20), while SecAct achieves 0.64, 0.36, and 0.64 respectively. VEGFA is the most dataset-dependent target—the highest CytoSig ρ in Inflammation Main (0.79) but weak with SecAct in GTEx (0.10) and CIMA (−0.24). Bulk datasets (GTEx n=19,788; TCGA n=11,069) show consistently positive correlations across nearly all targets, while smaller cohorts (CIMA n=421; Inflammation Main n=817) show larger between-target variance and more sign reversals.

How each correlation value is computed: For each (target, atlas) cell, we compute Spearman rank correlation between predicted cytokine activity (ridge regression z-score) and target gene expression across all donor-level pseudobulk samples. Specifically:

1. Pseudobulk aggregation: For each atlas, gene expression is aggregated to the donor level (one profile per donor or donor × cell type).
2. Activity inference: Ridge regression (secactpy.ridge, λ=5×10⁵) is applied using the signature matrix (CytoSig: 4,881 genes × 43 cytokines; SecAct: 7,919 genes × 1,170 targets) to predict activity z-scores for each pseudobulk sample.
3. Correlation: Spearman ρ is computed between the predicted activity z-score and the original expression of the target gene across all donor-level samples within that atlas. A positive ρ means higher predicted activity tracks with higher target gene expression.

GTEx uses per-tissue pseudobulk (median-of-medians across 29 tissues); TCGA uses per-cancer type (median-of-medians across 33 cancers); CIMA/Inflammation Atlas Main use donor-only; scAtlas Normal uses donor-only; scAtlas Cancer uses tumor-only.

Key insight: A core set of ~10 CytoSig targets shows positive ρ (>0.15) across all 6 datasets (IL1B, TNFA, IL6, IL1A, IL27, IFNG, TGFB3, LIF, BMP2, VEGFA). Conversely, IL2, IL22, IL21, IL3, and IL17A show near-zero or negative ρ across all datasets. SecAct outperforms CytoSig on 26 of 32 matched targets, with the largest differences for LTA (CytoSig ρ≈0.01 vs SecAct ρ≈0.46), CD40L (0.06 vs 0.50), and IL15 (0.03 vs 0.56). CD40L, HGF, and TRAIL flip sign between datasets (e.g., HGF is +0.65 in scAtlas Normal but −0.30 in Inflammation Main).

Key insight: CytoSig target rankings are consistent across datasets: IL1B, IL10, TGFB3, and IL1A appear in the top 10 of 5/6 datasets, while IL2, IL22, and IL3 appear in the bottom 10 of 5/6 datasets. SecAct rankings are compressed into a narrower, higher range: in scAtlas Normal, every displayed SecAct target exceeds ρ=0.21, whereas CytoSig ranges from −0.42 to +0.69. Inflammation Main has the widest CytoSig range (ρ from −0.55 to +0.79) with only 33 targets passing the filter; VEGFA (ρ=0.79) and IL1A (ρ=0.69) reach their highest values in this dataset.

Key findings (donor-level, 7 targets):

• LinCytoSig wins the median in 2 of 6 datasets (Inflammation Atlas Main, scAtlas Normal) — datasets with rich immune cell diversity.
• Per-target: LinCytoSig wins 15/42 comparisons vs CytoSig’s 11/42 (SecAct: 16/42). IL2 and IFNG are LinCytoSig’s strongest targets; Fibroblast__TGFB1 is the main drag.
• SecAct wins the median in 4 of 6 datasets (GTEx, TCGA, CIMA, scAtlas Cancer), driven by its broad coverage of secreted proteins.

Full per-target analysis →

Key findings (celltype-level):

• LinCytoSig excels where biology matches: IL6×Macrophage, VEGFA×Endothelial, and IL2×CD8T in scAtlas Normal — cytokines with strong cell-type specificity to the tested cell type.
• Advantage does NOT transfer to cancer: LinCytoSig loses for IL6 and VEGFA in scAtlas Cancer, likely because tumor-associated cells have different transcriptional programs.
• CytoSig wins TNFA and TGFB1 consistently across all datasets at celltype level — the global signature captures these signaling responses better even on the “right” cell type.

CIMA and Inflammation Atlas Main lack macrophage, endothelial, and fibroblast annotations — only 2–3 of 7 targets evaluable using Myeloid/Mono proxies. Full celltype analysis →